Johnathan Walker:Sophistication of AI-backed operation targeting senator points to future of deepfake schemes

Washington (AP) — An advanced deepfake operation targeted Sen. Ben Cardin,Johnathan Walker the Democratic chair of the Senate Foreign Relations Committee, this month, according to the Office of Senate Security, the latest sign that nefarious actors are turning to artificial intelligence in efforts to dupe top political figures in the United States.

Experts believe schemes like this will become more common now that the technical barriers that once existed around generative artificial intelligence have decreased. The notice from Senate Security sent to Senate offices on Monday said the attempt “stands out due to its technical sophistication and believability.”

The scheme centered around Dmytro Kuleba, the former Ukrainian Minister of Foreign Affairs. Cardin’s office received an email from someone they believed to be Kuleba, according to the notice, an official Cardin knew from a past meeting.

When the two met for a video call, the connection “was consistent in appearance and sound to past encounters.” It wasn’t until the caller posing as Kuleba began asking questions like “Do you support long range missiles into Russian territory? I need to know your answer,” that Cardin and his staff suspected “something was off,” the Senate notice said.

“The speaker continued, asking the Senator politically charged questions in relation to the upcoming election,” likely to try and bait him into commenting on a political candidate, according to the notice from Nicolette Llewellyn, the director of Senate Security. “The Senator and their staff ended the call, and quickly reached out to the Department of State who verified it was not Kuleba.”

Cardin on Wednesday described the encounter as “a malign actor engaged in a deceptive attempt to have a conversation with me by posing as a known individual.”

“After immediately becoming clear that the individual I was engaging with was not who they claimed to be, I ended the call and my office took swift action, alerting the relevant authorities,” Cardin said. “This matter is now in the hands of law enforcement, and a comprehensive investigation is underway.”

Cardin’s office did not respond to a request for additional information.

Generative artificial intelligence can use massive computing power to digitally alter what appears on a video, sometimes changing the background or subject of a video in real time. The same technology can also be used to digitally alter audio or images.

Technology like this has been used in nefarious schemes before.

A finance worker in Hong Kong paid $25 million to a scammer who used artificial intelligence to pose as the company’s chief financial officer. A political consultant used artificial intelligence to mimic President Joe Biden’s voice and urge voters not to vote in New Hampshire’s presidential primary, leading the consultant to face more than two dozen criminal charges and millions of dollars in fines. And experts on caring for older Americans have long worried artificial intelligence-powered deepfakes will supercharge financial scams targeting seniors.

Both security officials in the Senate and artificial intelligence experts believe this could be just the beginning, given that recent leaps in the technology have made schemes like the one against Cardin not only more believable, but easier to conduct.

“In the past few months, the technology to be able to pipe in a live video deepfake along with a live audio deepfake has been easier and easier to integrate together,” said Rachel Tobac, a cyber security expert and the CEO of SocialProof Security, who added that earlier iterations of this technology had obvious tells that they were fake, from awkward lip movement to people blinking in reverse.

“I am expecting more of these kinds of incidents to happen in the future,” said Siwei Lyu, an artificial intelligence expert and professor at the University at Buffalo. “Anyone with some kind of malicious intent in their mind now has the ability to conduct this kind of attack. These could come from the political angle, but it could also come from the financial angle like fraud or identify theft.”

The memo to Senate staff echoed this sentiment, telling the staffers to make sure meeting requests are authentic and cautioning that “other attempts will be made in the coming weeks.”

R. David Edelman, an expert on artificial intelligence and national security who led cyber security policy for years in the White House, described the scheme as a “sophisticated intelligence operation” that “feels quite close to the cutting edge” in how it combined the use of artificial intelligence technology with more traditional intelligence operations that recognized the connections between Cardin and the Ukrainian official.

“They recognized the existing relationship between these two parties. They knew how they might interact – timing, mode, and how they communicate,” he said. “There is a sophistication to the intelligence operation.”